Methods and systems for automated migration of cloud processes to external clouds

ABSTRACT

A cloud management system can receive rules for altering the virtual machines based on demands on the virtual machines and/or computing resources supporting the virtual machines. The cloud management system can receive data from the internal monitoring agents and/or external monitoring agents and to determine when the conditions of the rules are met. Once the conditions are met, the cloud management system can migrate the virtual machines to an external cloud.

FIELD

This invention relates generally to network computing, moreparticularly, to systems and methods for cloud computing relatednetworks, services, and products.

DESCRIPTION OF THE RELATED ART

The advent of cloud-based computing architectures has opened newpossibilities for the rapid and scalable deployment of virtual Webstores, media outlets, and other on-line sites or services. In general,a cloud-based architecture deploys a set of hosted resources such asprocessors, operating systems, software and other components that can becombined or strung together to form virtual machines. A user or customercan request the instantiation of a virtual machine or set of machinesfrom those resources from a central server or management system toperform intended tasks or applications. For example, a user may wish toset up and instantiate a virtual server from the cloud to create astorefront to market products or services on a temporary basis, forinstance, to sell tickets to an upcoming sports or musical performance.The user can lease or subscribe to the set of resources needed to buildand run the set of instantiated virtual machines on a comparativelyshort-term basis, such as hours or days, for their intended application.

Currently, when a requester acquires access to the cloud for a virtualmachine, the virtual machine is assigned fixed computing resources.Typically, the cloud architecture does not consider changes in thedemand and usage of the virtual machines. As such, the requester lacksthe ability to flexibly request use of the computing resources based onthe demand on and the usage of the virtual machines. Additionally, thevirtual machine is limited to the computing resources contained in thecloud.

BRIEF DESCRIPTION OF THE DRAWINGS

Various features of the embodiments can be more fully appreciated, asthe same become better understood with reference to the followingdetailed description of the embodiments when considered in connectionwith the accompanying figures, in which:

FIG. 1 illustrates an exemplary cloud computing architecture in whichvarious embodiments of the present teachings can be practiced;

FIG. 2 illustrates the exemplary cloud computing architecture in which acloud management system can utilize a cloud master and migrate virtualmachines to external clouds, according to various embodiments;

FIG. 3 illustrates an exemplary hardware configuration for a cloudmanagement system, according to various embodiments; and

FIG. 4 illustrates a flowchart of an exemplary process for migratingvirtual machines to external clouds, according to various embodiments.

DETAILED DESCRIPTION OF EMBODIMENTS

For simplicity and illustrative purposes, the principles of the presentteachings are described by referring mainly to exemplary embodimentsthereof. However, one of ordinary skill in the art would readilyrecognize that the same principles are equally applicable to, and can beimplemented in, all types of information and systems, and that any suchvariations do not depart from the true spirit and scope of the presentteachings. Moreover, in the following detailed description, referencesare made to the accompanying figures, which illustrate specificembodiments. Electrical, mechanical, logical and structural changes maybe made to the embodiments without departing from the spirit and scopeof the present teachings. The following detailed description is,therefore, not to be taken in a limiting sense and the scope of thepresent teachings is defined by the appended claims and theirequivalents.

Embodiments of the present teachings relate to systems and methods forflexible management of a cloud computing environment. More particularly,embodiments relate to platforms and techniques in which a cloudmanagement system can allow requesters to define rules to alter virtualmachines such as migrating virtual machines to external clouds based onthe demand on and usage of the cloud.

According to embodiments, the cloud management system can be configuredto include a cloud master. The cloud master can be configured to providean interface for the cloud management system or a requester to specifyrules for altering the virtual machines based on demands on the virtualmachines and/or computing resources supporting the virtual machines. Therules can specify conditions in usage of and demands on the instantiatedvirtual machines and/or the computing resources when the virtualmachines should be altered. The alteration can include migrating thevirtual machines to an external cloud once the usage of or demands onthe virtual machines and/or computing resources meet the specifiedconditions.

According to embodiments, the cloud master can be configured to receivethe rules in a universal format that can be utilized by the requesters.The cloud master can be configured to receive the rules in a naturaltext-based language. Additionally, the cloud master can receive rulesvia a variety of communication channels and protocols.

According to embodiments, the cloud master can also be configured toinclude application programming interfaces (APIs) to allow the rules tolinked to internal monitoring agents and/or external monitoring agents.The internal monitoring agents can be tools and plug-ins provided by thecloud management system to monitor the usage of and the demands on thecomputing resources and/or the instantiated virtual machines. Theexternal monitoring agents can be systems, tools, and plug-ins, separatefor the cloud management system, to monitor the usage of and demands onthe instantiated virtual machines and/or the computing resources.

According to embodiments, once the rules are received, the cloud mastercan be configured to receive data from the internal monitoring agentsand/or external monitoring agents to determine when the conditions ofthe rules are met. Once the conditions are met, the cloud master can beconfigured to notify the cloud management system that the conditions aremet and the action to be taken, for example, migrating the one or morevirtual machines to an external cloud. Accordingly, the cloud managementsystem can be configured to cooperate with an independent cloudmanagement system of the external cloud in order to migrate the one ormore virtual machines.

By allowing rules for altering virtual machines, the cloud managementsystem can enable requesters to tailor their usage of the cloud in orderto address changing conditions in the virtual machines. Additionally, byreceiving the rules in a universal format, the cloud management systemcan extend cloud management processes to any type of requester.Moreover, because the cloud management system can migrate virtualmachines to external clouds, the cloud management system can provide anexpanded amount of computing resources to requesters. Thus, the cloudmanagement system can provide flexibility and efficiency to any cloudcomputing environment.

FIG. 1 illustrates an overall cloud computing environment 100, in whichsystems and methods for the flexible management of the cloud computingenvironment 100, according to embodiments of the present teachings.According to embodiments, a cloud management system 102 can beconfigured to mange one or more clouds, such as a dedicated cloud 104and an ad-hoc cloud 106. As used herein, a “cloud” can comprise acollection of computing resources that can be invoked to instantiate avirtual machine, process, or other resource for a limited or definedduration.

As shown for example in FIG. 1, the collection of computing resourcessupporting the dedicated cloud 104 can comprise a set of resourceservers 108 configured to deliver computing resources and componentsneeded to instantiate a virtual machine, process, or other resource. Forexample, one group of resource servers can host and serve an operatingsystem or components thereof to deliver to and instantiate a virtualmachine. Another group of resource servers can accept requests to hostcomputing cycles or processor time, to supply a defined level ofprocessing power for a virtual machine. A further group of resourceservers can host and serve applications to load on an instantiation of avirtual machine, such as an email client, a browser application, amessaging application, or other applications or software. Other types ofresource servers are possible.

In embodiments, in addition to supporting the dedicated cloud 104, thecloud management system 102 can be configured to support the ad-hoccloud 106. The ad-hoc cloud 106 can be composed of a variety ofcomputing resources that may not be dedicated to a cloud but can haveavailable computing resources to contribute to the ad-hoc cloud 106. Forexample, a corporation or university can have a large number ofcomputing resources that support a variety of process (email, websites,individual user computing, and the like). The corporation or universitycan utilize the available excess computing resources to support anad-hoc cloud, such as ad-hoc cloud 106.

In embodiments, as shown in FIG. 1, the ad-hoc cloud 106 can besupported by a number of computing systems 110. For example, thecomputing systems 110 can include a variety of systems such as a set ofservers 112 and 114 and standalone user computing systems 116 and 118.The computing systems 110 can include hardware resources, such asprocessors, memory, network hardware, storage devices, and the like, andsoftware resources, such as operating systems (OS), applicationprograms, and the like.

In embodiments, the entire set of resource servers 108 or other hardwareor software resources used to support the cloud 104 and the computingsystems 110 used to support the cloud 106 can be managed by the cloudmanagement system 102. The cloud management system 102 can comprise adedicated or centralized server and/or other software, hardware, andnetwork tools that communicate via one or more networks 120 and networks122, such as the Internet or other public or private network, with allsets of resource servers 108 to manage the cloud 104 and with computingsystems 110 to manage the cloud 106 and their operation.

In embodiments, to manage the clouds 104 and 106, the cloud managementsystem 102 can be configured identify the computing resources of the setof resource servers 108 and computing systems 110. The cloud managementsystem 102 can be configured to include a network management agent thatis capable of querying the set of resource servers 108 and computingsystems 110 to determine the hardware and software resources. Likewise,the cloud management system 102 can be configured to communicate withexternal network management systems and/or resources monitoring agentsexecuting on the set of resource servers 108 and computing systems 110in order to determine the hardware and software resources of the set ofresource servers 108 and computing systems 110.

In embodiments, the cloud management system 102 can be configured toidentify both the hardware and software resources of the set of resourceservers 108 and computing systems 110 and which of those resources areavailable for use in the cloud. The cloud management system 102 can beconfigured to identify the hardware resources such as type and amount ofprocessing power, type and amount of memory, type and amount of storage,type and amount of network bandwidth and the like, of the set ofresource servers 108 and computing systems 110. Likewise, the cloudmanagement system can be configured to identify the software resources,such as type of OS, application programs, and the like, of the set ofresource servers 108 and computing systems 110.

In embodiments, once the computing resources have been identified, thecloud management system 102 can be configured to store an identificationof the available resources in an inventory 124 in a repository 126. Therepository 126 can be any type of structure configured to storeinformation, such as a database. The repository 126 can be maintained ina computer readable storage device or medium whether local to or remotefrom the cloud management system 102.

In embodiments, the inventory 124 can be configured to includeinformation that identifies the set of resource servers 108 andcomputing systems 110 and information identifying the computingresources available. The sets of resource servers 108 and each system inthe computing systems 110 can be identified by unique identifiers suchas, for instance, Internet Protocol (IP) addresses or other addresses.In the inventory 124, the cloud management system 102 can associate,with each unique identifier, the computing resources available on thatcomputing system.

In embodiments, to instantiate a new set of virtual machines, arequester can transmit an instantiation request to the cloud managementsystem 102. The instantiation request can include the specifications forthe set of virtual machines. The specifications can include theparticular type of virtual machine they wish to invoke for theirintended application. A requester can, for instance, make a request toinstantiate a set of virtual machines configured for email, messaging orother applications from the cloud 104 and/or 106. The specifications canalso include the type and/or amount of computing resources required. Forexample, the instantiation request can specify an amount of processingpower or input/output (I/O) throughput the user wishes to be availableto each instance of the virtual machine or other resources.

In embodiments, the requester's instantiation request can specify avariety of other specifications defining the configuration and operationof the set of virtual machines to be invoked. The instantiation request,for example, can specify a defined period of time for which theinstantiated machine or process is needed. The period of time can be,for example, an hour, a day, or other increment of time. In embodiments,the requester's instantiation request can specify the instantiation of aset of virtual machines or processes on a task basis, rather than for apredetermined amount of time. For instance, a requester could requestresources until a software update is completed. The requester can also,for instance, specify a service level agreement (SLA) acceptable fortheir application. One skilled in the art will realize that therequester's request can likewise include combinations of the foregoingexemplary specifications, and others.

In embodiments, the instantiation request can be received and processedby the cloud management system 102, which identifies the type of virtualmachine, process, or other resource being requested from thespecifications. The cloud management system 102 can then identify thecollection of computing resources necessary to instantiate that machineor resource. For example, the set of instantiated virtual machines orother resources can for example comprise virtual transaction serversused to support Web storefronts, or other transaction sites.

In embodiments, the cloud management system 102 can be configured toutilize the specifications from the instantiation request and theinventory 124 of available computing resources to determine which cloudresources to devote to the requester's virtual machines to maximize thecomputing resources of the clouds 104 and/or 106 and meet therequester's specifications. For example, the cloud management system 102can select a group of servers in the set of resource servers 108 and/orcomputing system in the computing systems 110 that match or best matchthe instantiation request for each component needed to build the virtualmachine or other resource.

In embodiments, the cloud management system 102 can maintain a set of“virtual groups,” and assign the set of resource servers 108 andcomputing systems 110 to different “virtual groups”. The “virtualgroups” can be based on the particular usage (type of virtual machine,application of the virtual machine, function of the virtual machine, andthe like) of the members in the groups. For example, the cloudmanagement system 102 can set up a “virtual group” for web servers. Thecloud management system 102 can classify the computing resources for theweb server “virtual group” based on which computing resources are bestsuited for web servers. As members of the web server “virtual group”request use of the cloud, the cloud management system 102 can assign theavailable computing resources classified in the web server “virtualgroup” to the members. Likewise, the “virtual groups” can be based onthe specifications of the computing resources (type and amount ofcomputing resources). For example, the cloud management system 102 cancreate a “virtual group” for high power computing users. The cloudmanagement system 102 can assign resources to this group that canadequately support computing intensive virtual machines. As members ofthe high power “virtual group” request use of the cloud, the cloudmanagement system 102 can assign the available computing resourcesclassified in the high power “virtual group” to the members. The cloudmanagement system 102 can maintain the virtual groups in a group record128 in repository 126.

When the request to instantiate a set of virtual machines or otherresources has been received and the necessary resources to build thatmachine or resource have been identified, the cloud management system102 can communicate with one or more set of resource servers 108 and/orcomputing systems 110 to locate resources to supply the requiredcomponents. The cloud management system 102 can select providers fromthe diverse set of resource servers 108 and/or computing systems 110 toassemble the various components needed to build the requested set ofvirtual machines or other resources. It may be noted that in someembodiments, permanent storage such as hard disk arrays may not beincluded or located within the set of resource servers 108 and thecomputing resources 110 available to the cloud management system 102,because the set of instantiated virtual machines or other resources maybe intended to operate on a purely transient or temporary basis. Inembodiments, other hardware, software or other resources not strictlylocated or hosted in the cloud can be leveraged as needed. For example,other software services that are provided outside of the clouds 104 and106 and hosted by third parties can be invoked by in-cloud virtualmachines. For further example, other non-cloud hardware and/or storageservices can be utilized as an extension to the clouds 104 and 106,either on an on-demand or subscribed or decided basis.

With the specification and resources identified, the cloud managementsystem 102 can extract and build the set of virtual machines or otherresources on a dynamic or on-demand basis. For example, one set ofresource servers 108 or computing systems 110 can respond to aninstantiation request for a given quantity of processor cycles with anoffer to deliver that computational power immediately and guaranteed forthe next hour. A further set of resource servers 108 or computingsystems 110 can offer to immediately supply communication bandwidth, forexample on a guaranteed minimum or best-efforts basis. In otherembodiments, the set of virtual machines or other resources can be builton a batch basis or at a particular future time. For example, a set ofresource servers 108 and/or computing systems 110 can respond to arequest for instantiation at a programmed time with an offer to deliverthe specified quantity of processor cycles within a specific amount oftime, such as the next 12 hours.

In embodiments, the cloud management system 102 can then coordinate theintegration of the completed group of servers from the set of resourceservers 108 and/or computing systems from the computing systems 110, tobuild and launch the requested set of virtual machines or otherresources. The cloud management system 102 can track the combined groupof servers selected from the set of resource servers 108, computingsystems from the computing systems 110, or other distributed resourcesthat are dynamically or temporarily combined, to produce and manage therequested virtual machine population or other resources.

In embodiments, the cloud management system 102 can then set up andlaunch the initiation process for the virtual machines, processes, orother resources to be delivered from the cloud. The cloud managementsystem 102 can for instance transmit an instantiation command orinstruction to the group of servers in set of resource servers 108and/or computing system in the computing systems 110. The cloudmanagement system 102 can receive a confirmation message back from eachparticipating server in a set of resource servers 108 and/or computingsystem in the computing systems 110 indicating a status regarding theprovisioning of their respective resources. Various sets of resourceservers can confirm, for example, the availability of a dedicated amountof processor cycles, amounts of electronic memory, communicationsbandwidth, or applications or other software prepared to be served.

In embodiments, the cloud management system 102 can maintain a VM record130 of each virtual machine instantiated in the clouds 104 and 106. Eachvirtual machine can be assigned an instantiated machine ID that can bestored in the VM record 130, or other record or image of theinstantiated population. Additionally, the cloud management system 102can store the duration of each virtual machine and the collection ofresources utilized by each virtual machine in the VM record 130 and/orinventory 124. The cloud management system 102 can maintain the VMrecord 130 in the repository 126.

In embodiments, the cloud management system 102 can further store, trackand manage a requester's identity and associated set of rights orentitlements to software, hardware, and other resources. Each requesterthat populates a set of virtual machines in the cloud can have specificrights and resources assigned and made available to them. The cloudmanagement system 102 can track and configure specific actions that arequester can perform, such as provision a set of virtual machines withsoftware applications or other resources, configure a set of virtualmachines to desired specifications, submit jobs to the set of virtualmachines or other host, manage other requesters of the virtual machinesor other resources, and other privileges or actions. The cloudmanagement system 102 can further generate records of the usage ofinstantiated virtual machines to permit tracking, billing, and auditingof the services consumed by the requester. In embodiments, the cloudmanagement system 102 can for example meter the usage and/or duration ofthe virtual machines, to generate subscription billing records for arequester that has launched those machines. Other billing or valuearrangements are possible.

The cloud management system 102 can configure each virtual machine to bemade available to requester and/or users of the one or more networks 120and/or 122 via a browser interface, or other interface or mechanism.Each instantiated virtual machine can communicate with the cloudmanagement system 102 and the underlying registered set of resourceservers 108 and/or computing systems 110 via a standard Web applicationprogramming interface (API), or via other calls or interfaces. Theinstantiated virtual machines can likewise communicate with each other,as well as other sites, servers, locations, and resources available viathe Internet or other public or private networks, whether within a givencloud 104 or 106 or between clouds.

It may be noted that while a browser interface or other front-end can beused to view and operate the instantiated virtual machines from a clientor terminal, the processing, memory, communications, storage, and otherhardware as well as software resources required to be combined to buildthe virtual machines or other resources are all hosted remotely in theclouds 104 and 106. In embodiments, the virtual machines or otherresources may not depend on or require the requester's own on-premisehardware or other resources. In embodiments, a requester can thereforerequest and instantiate a set of virtual machines or other resources ona purely off-premise basis, for instance to build and launch a virtualstorefront or other application.

Because the cloud management system 102 in one regard specifies, builds,operates and manages the virtual machines on a logical level, therequester can request and receive different sets of virtual machines andother resources on a real-time or near real-time basis, without a needto specify or install any particular hardware. The requester's virtualmachines, processes, or other resources can be scaled up or downimmediately or virtually immediately on an on-demand basis, if desired.In embodiments, the various sets of computing resources that areaccessed by the cloud management system 102 to support the virtualmachines or processes can change or be substituted, over time. The typeand operating characteristics of the virtual machines can neverthelessremain constant or virtually constant, since instances are assembledfrom abstracted resources that can be selected and maintained fromdiverse sources based on uniform specifications.

In terms of network management of the virtual machines that have beensuccessfully configured and instantiated, the cloud management system102 can perform various network management tasks including security,maintenance, and metering for billing or subscription purposes. Thecloud management system 102 of a given cloud 104 or 106 can, forexample, install or terminate applications or appliances on individualmachines. The cloud management system 102 can monitor operating virtualmachines to detect any virus or other rogue process on individualmachines, and for instance terminate the infected application or virtualmachine. The cloud management system 102 can likewise manage the virtualmachines or other resources on a collective basis, for instance, to pushor deliver a software upgrade to all active virtual machines. Othermanagement processes are possible. Likewise, the cloud management system102 can be configured to communicate with external network managementsystems to coordinate the network management functions and processes.

In embodiments, more than one set of virtual machines can beinstantiated in a given cloud at the same, overlapping or successivetimes. The cloud management system 102 can, in such implementations,build, launch and manage multiple sets of virtual machines based on thesame or different underlying set of resource servers 108 or computingsystems 110, with populations of different sets of virtual machines suchas may be requested by different requesters. The cloud management system102 can institute and enforce security protocols in the clouds 104 and106 hosting multiple sets of virtual machines. Each of the individualsets of virtual machines can be hosted in a respective partition orsub-cloud of the resources of the clouds 104 and/or 106. The cloudmanagement system 102 of a cloud can for example deploy servicesspecific to isolated or defined sub-clouds, or isolate individualworkloads/processes within the cloud to a specific sub-cloud. Thesubdivision of the clouds 104 and/or 106 into distinct transientsub-clouds or other sub-components which have assured security andisolation features can assist in establishing multiple requesters or amulti-tenant cloud arrangement. In a multiple requesters scenario, eachof the multiple requesters can use the cloud platform as a commonutility while retaining the assurance that their information is securefrom other requesters of the overall cloud system. In furtherembodiments, sub-clouds can nevertheless be configured to shareresources, if desired.

In embodiments, the instantiated virtual machines supported by the cloud104 can also interact with instantiated virtual machines or processesgenerated in the cloud 106 or other clouds and vice versa. The cloudmanagement system 102 of clouds 104 and 106 can interface with the cloudmanagement system of other clouds, to coordinate those domains andoperate the clouds and/or virtual machines or processes on a combinedbasis.

As described above, the cloud management system 102 can instantiate andmanage the virtual machines instantiated in the clouds 104 and 106. Inembodiments, the instantiation and management of virtual machines can beperformed by virtual machine (VM) managers separate from the cloudmanagement system 102. The cloud management system 102 can be configuredto communicate with the separate VM managers in order to provide the VMmanagers with the computing resources available in the clouds 104 and106. The cloud management system 102 can be configured to communicateand cooperate with the VM managers regardless of the virtualizationscheme used by the VM managers.

In the foregoing and other embodiments, the requester making aninstantiation request or otherwise accessing or utilizing the cloudnetwork can be a person, customer, subscriber, administrator,corporation, organization, or other entity. In embodiments, therequester can be or include another virtual machine, application orprocess. In further embodiments, multiple requesters and/or entities canshare the use of a set of virtual machines or other resources.

FIG. 2 further illustrates aspects of the cloud computing environment100 in which the cloud management system 102 can manage the dedicatedcloud 104 and the ad-hoc cloud 106 utilizing a cloud master 200,according to various embodiments. While FIG. 2 only illustrates theinteraction of cloud management system 102 with the dedicated cloud 104and the ad-hoc cloud 106, one skilled in the art will realize that thecloud management system 102 can manage any number of clouds, forinstance, only one of the dedicated cloud 104 and the ad-hoc cloud 106or other clouds in addition to the dedicated cloud 104 and the ad-hoccloud 106.

As shown in FIG. 2, the cloud management system 102 can be coupled to anetwork 120 to communicate with the set of resource servers 108 andcoupled to the network 122 to communicate with computing systems 110 toprovide management services for the dedicated cloud 104 and the ad-hoccloud 106. As mentioned above, the dedicated cloud 104 can comprise aset of resource servers 108 configured to deliver computing resourcesand components needed to instantiate a virtual machine, process, orother resource. The ad-hoc cloud 106 can be composed of a variety ofcomputing resources that may not be dedicated to a cloud but can haveavailable computing resources to contribute to the ad-hoc cloud 106. Forexample, a corporation or university can have a large number ofcomputing resources that support a variety of processes (email,websites, individual user computing, and the like). The corporation oruniversity can utilize the available excess computing resources tosupport the ad-hoc cloud 106.

In embodiments, as shown in FIG. 2, the ad-hoc cloud 106 can besupported by the computing systems 110. For example, the computingsystems 110 can include a variety of systems such as a set of servers112 and 114 and standalone user computing systems 116 and 118. Thecomputing systems 110 can include hardware resources, such asprocessors, memory, network hardware, storage devices, and the like, andsoftware resources, such as operating systems (OS), applicationprograms, and the like.

In embodiments, as described above in FIG. 1, the cloud managementsystem 102 can be configured to instantiate virtual machines 202 in thededicated cloud 104 and/or the ad-hoc cloud 106. The virtual machines202 can be configured to perform a variety of processes, services,tasks, and the like. For example, the virtual machines 202 can besupporting the different components of a web site, e.g. webserver,database server, etc. When the virtual machines 202 are requested, therequesters can specify the computing resources required for the virtualmachines 202. For instance, in the above web site example, the requestercan specify hardware resources (an amount of bandwidth, processingpower, memory, etc.) and software resources (types of web serverapplication, database application, etc.) to support and comprise thevirtual machines 202. The requester can specify these computingresources based on the expected usage of the computing resourcessupporting the virtual machines 202 (e.g. bandwidth to support expectedtraffic on the web site). The cloud management system 102 can beconfigured to identify and selected the set of resource servers 108and/or the computing systems 110 that meet the requested computingresources and instantiate the virtual machines 202 on the selected setof resource servers 108 and/or the computing systems 110.

In embodiments, the demands on the virtual machines 202 instantiated inthe dedicated cloud 104 and/or the ad-hoc cloud 106 can fluctuate overtime. As such, the virtual machines' usage (increased or reduced) of thecomputing resources of the set of resource servers 108 and/or computingsystems 110 can vary over time. For example, virtual machines 202supporting a shopping website can experience a large increase in trafficduring certain time periods, e.g. Christmas season. In some cases, thevirtual machines 202 usage of the computing resources can reach thelimits of the computing resources supporting them or significantlyunder-utilize the computing resources. For example, virtual machines 202supporting the shopping website can experience a large increase intraffic during the Christmas season and can reach the limit of thecomputing resources, for instance, network bandwidth, of the set ofresource servers 108 and/or the computing systems 110 originallyallocated for the website, thereby causes slow access to the shoppingwebsite. In contrast, immediately after Christmas, the virtual machines202 supporting the shopping website can experience a sharp decrease inusage and have significant excess in computing resources.

In embodiments, to provide flexibility in the computing resourcessupporting the virtual machines 202, the cloud management system 102 canbe configured to include the cloud master 200. The cloud master 200 canbe configured to provide an interface 204 for the cloud managementsystem 102 or requester to specify rules for altering the virtualmachines 202 based on demands on the virtual machines and/or computingresources of the set of resource servers 108 and/or computing systems110. In particular, the rules can specify conditions in usage of anddemands on the instantiated virtual machines 202, the set of resourceservers 108, and/or the computing systems 110 when the virtual machines202 should be altered. Additionally, the rules can specify actions totake once the virtual machines 202 and/or computing resources meet thespecified conditions. The actions can include altering the virtualmachines 202 and/or operating state of the virtual machines 202, such asany adding new virtual machines 202, removing one or more of the virtualmachines 202, adding computing resources to the virtual machines 202,removing computing resources from the virtual machines 202, migratingthe virtual machines 202 to one or more of the set of resource servers108, and/or the computing systems 110 with additional computingresources, migrating the virtual machines 202 to one or more of the setof resource servers 108, and/or the computing systems 110 with lesscomputing resources, and the like.

In embodiments, the cloud master 200, via the interface 204, can beconfigured to receive the rules in a universal format that can beutilized by the requesters. For example, the cloud master 200 can beconfigured to receive the rules in a natural text-based language.Additionally, the cloud master 200, via the interface 204, can receiverules via a variety of communication channels and protocols. Theinterface 202 can be configured to supporting a scripting interface toallow the rules to be entered in the universal format. Likewise, theinterface 202 can be configured to receive the rules in via otherchannels and protocols, such as electronic mail (email), simple networkmanagement protocol (SNMP), web service, message bus, and the like.

In embodiments, the cloud master 200 can also be configured to includeapplication programming interfaces (APIs) to allow the rules to linkedto internal monitoring agents 206 and/or external monitoring agents 208.The internal monitoring agents 206 can be tools and plug-ins provided bythe cloud management system 102 to monitor the usage of and the demandson the set of resource servers 108, the computing systems 110, and/orthe instantiated virtual machines 202. For example, the internalmonitoring agents 206 can be conventional network monitoring tools,conventional computing resource monitoring tools, conventional virtualmachine monitoring tools, and the like which have been modified tomonitor the dedicated cloud 104, the ad-hoc cloud 106, and theinstantiated virtual machines 202.

In embodiments, the external monitoring agents 208 can be systems,tools, and plug-ins, separate for the cloud management system 102, tomonitor the usage of and demands on the instantiated virtual machines202, the set of resource servers 108, and/or the computing systems 110.For example, the external monitoring agents 208 can be conventionalnetwork monitoring tools, conventional computing resource monitoringtools, conventional virtual machine monitoring tools, and the like.

In embodiments, once the rules are received, the cloud master 200 can beconfigured to receive data from the internal monitoring agents 204and/or external monitoring agents 206 and to determine when theconditions of the rules are met. Once the conditions are met, the cloudmaster 200 can be configured to notify the cloud management system 102that the conditions are met and the action to be taken. Accordingly, thecloud management system 102 can be configured to take the appropriateaction to alter the instantiated virtual machines 202.

In embodiments, the cloud master 200 can be configured to include thenecessary logic, routines, instruction, and commands to communicate withthe cloud management system 102 and to provide the interface asdescribed above. The cloud master 200 can be implemented as a portion ofthe code for the cloud management system 102. Likewise, the cloud master200 can be implemented as a separate software tool accessible by thecloud management system 102. The cloud master 200 can be written in avariety of programming languages, such as JAVA, C++, Python code, andthe like to accommodate a variety of operating systems, machinearchitectures, etc. Additionally, the cloud master 200 can be configuredto include the appropriate APIs to communicate with and cooperate withother components of the cloud management system 102 and to provide theinterface as described above.

In one example, a requester 210 can request and the cloud managementsystem 102 can instantiate a virtual machine 212 in the dedicated cloud104. The virtual machine 212 can be supporting the website “PoliticalNews” which provides news about national politics. When requesting theuse of the cloud 104, the requester 212 can specify a particularcomputing resources requirement to meet the average expected traffic orwait time on the website. Accordingly, the cloud management system 102can select the set of resource servers 108 to meet the computingresources requirements.

In this example, the requester 210 may know that “Political News” canexperience a spike in traffic on the website at certain times, forinstance, during election season or when a particular political newsstory occurs, which can cause an increase in wait time. To account forthe expected spike in traffic, the requester 210 can desire to increasethe computing resources or add new virtual machines supporting thewebsite during these times. As such, the requester 210 can utilize thecloud master 200, via the interface 204, to create a rule to meet thisexpected spike. For example, the request 210 can specify a rule thatstates: when the traffic on “Political News” reaches a certain level,increase the bandwidth for the virtual machine 212. Using the universalformat, the rule can take the form “—system virtual machine 212—waittime>2 seconds—increase computing resources”.

Additionally, in this example, the requester 210 can specify theinternal monitoring agent 206 and/or external monitoring agent 208 thatwill be monitoring the traffic or wait time on the website. Forinstance, the requester 210 can have access to an external monitoringagent 214 for monitoring traffic or wait time on a website. Therequester 210 can utilize the interface 204 to specify, in the rule, theexternal monitoring agent 214 that will be monitoring the traffic orwait time on “Political News”.

In this example, once the rule has been specified, the cloud master 200can receive traffic or wait time data from the external monitoring agent214 and determine if the traffic or wait time data meets the conditionspecified in the rule. If the wait time reaches 2 seconds, the cloudmaster 200 can notify the cloud management system 102 that the virtualmachine 212 needs additional computing resources. Accordingly, the cloudmanagement system 102 can take an appropriate action to alter thevirtual machine 212.

In this example, to increase the computing resources, the cloudmanagement system 102 can reconfigure the virtual machine 212 to utilizemore of the computing resources of the set of resource servers 108.Likewise, the cloud management system 102 can migrate the virtualmachine 212 to other of the set of resource servers 108 in the dedicatedcloud 104 or migrate the virtual machine 212 to the computing systems110 in the ad-hoc cloud 106. Likewise, the cloud management system 102can add additional computing resources from the set of resource servers108 and/or computing systems 110 to the virtual machine 212. In order todetermine which of the set of resource servers 108 and/or computingsystems 110 are available, the cloud management system 102 can examinethe inventory 124.

The above example describes an exemplary situation for altering thevirtual machine 212 upon the occurrence of a condition. One skilled inthe art will realize that the requester 210 can specify any rules andconditions for altering the virtual machine 212 using the cloud master200. For instance, the requester 210 can specify rules for decreasingthe computer resources for the virtual machine 212 when traffic or waittime on the website decreases. Likewise, the requester 210 can specifyany conditions related to the virtual machine 212 and the computingresources for altering the virtual machine 212. Additionally, therequester 210 can specify specific action to be taken (e.g. migratingthe virtual machine 212).

In embodiments, the cloud master 200, via the interface 204, can beconfigured to receive instructions for alter virtual machines 202,immediately, instead of upon the occurrence of the condition. Forexample, the cloud master 200 can receive an instruction to immediatelyincrease the computing resources, migrate the virtual machines 202 toone or more of the set of resource servers 108, and/or the computingsystems 110 with additional computing resources, migrating the virtualmachines 202 to one or more of the set of resource servers 108, and/orthe computing systems 110 with less computing resources, and the like.

In embodiments, the cloud management system 102 can be configured tocommunicate with an external cloud 250 controlled by an independentcloud management system 252, via one or more networks 254. The externalcloud 250 can be any type of cloud computing environment which is underthe control of the independent cloud management system 252. The externalcloud 250 can include its own computing resources 256 under the controlof the independent cloud management system 252. For example, theexternal cloud 250 can be a commercially available cloud maintained by aprovider, such as EC2 maintained by AMAZON, and/or cloud maintained byan separate and independent entity.

In embodiments, the one or more networks 254 can be or include theInternet, or other public or private networks. The one or more or morenetworks 254 can be or include wired, wireless, optical, and othernetwork connections. One skilled in the art will realize that the one ormore networks 254 can be any type of network, utilizing any type ofcommunication protocol, to connect the computing systems.

In embodiments, as mentioned above, when the conditions of a particularrule are met, the cloud master 200 can be configured to notify the cloudmanagement system 102. In response to the notification, the cloudmanagement system 102 can be to take the appropriate or specifiedaction, such as migrating one or more of the virtual machines. Whenmigrating the virtual machines, the cloud management system 102 can beconfigured to migrate the virtual machines to the computing system 256of the external cloud 250.

For example, as mentioned above, the requester 210 can utilize the cloudmaster 200, via the interface 204, to create a rule to meet an expectedspike in traffic on the website “Political News”. For example, therequest 210 can specify a rule that states: when the traffic on“Political News” reaches a certain level, increase the bandwidth for thevirtual machine 212. Using the universal format, the rule can take theform “—system virtual machine 212—wait time>2 seconds—increase computingresources”. If the condition occurs, the cloud management system 102 maydetermine that the dedicated cloud 104 and the ad-hoc cloud 106 do nothave the computing resources to meet the conditions specified in therule. As such, the cloud management system 102 can be configured tomigrate the virtual machine 212 to the external cloud 250, in order tosupplement the computing resources provided and meet the conditions ofthe rule.

In embodiments, in order to migrate the virtual machines, the cloudmanagement system 102 can be configured to communicate with theindependent cloud management system 252 in order to determine theavailable computing resources 256 in the external cloud 250 and theconfiguration management scheme (e.g. virtualization scheme) utilized bythe independent cloud management system 252 to instantiate andcommunicate with virtual machines. The cloud management system 102 canbe configured to communicate with the independent cloud managementsystem 252, periodically, and/or at the time the virtual machines needto be migrated to the external cloud 250. Likewise, once determined, thecloud management system 102 can be configured to store a record of theavailable computing resources 256 and the cloud management scheme in theinventory 124.

In embodiments, when migrating the virtual machines to the externalcloud 250, the cloud management system 102 can directly instantiate thevirtual machines on the computing resources 256 of the external cloud250. For instance, the cloud management system 102 can communicate withthe independent cloud management system 252 and the independent cloudmanagement system 252 can provide the identification and information forthe computing resources 256 that will support the virtual machines andthe cloud management scheme utilized to instantiate virtual machines.Additionally, for instance, the cloud management system 102 can searchthe inventory 124 for the record of the available computing resources256 and the cloud management scheme. As such, the cloud managementsystem 102 can directly instantiate the virtual machines on thecomputing resources 256, as described above. Likewise, for example, thecloud management system 102 can provide the virtual machines and/ordetails of the virtual machines to the independent cloud managementsystem 252, and the independent cloud management system 252 caninstantiate the virtual machine.

FIG. 3 illustrates an exemplary diagram of hardware and other resourcesthat can be incorporated in a computing system 300 and configured tocommunicate with the clouds 104 and 106 via one or more networks 120 and122 and communicate with the external cloud 250 via the network 254, tosupport the cloud management system 102, according to embodiments. Inembodiments as shown, the computing system 300 can comprise a processor302 communicating with memory 304, such as electronic random accessmemory, operating under control of or in conjunction with operatingsystem 308. Operating system 308 can be, for example, a distribution ofthe Linux™ operating system, such as SELinux, the Unix™ operatingsystem, or other open-source or proprietary operating system orplatform. Processor 302 also communicates with one or more computerreadable storage devices or media 310, such as hard drives, opticalstorage, and the like, for maintaining the repository 126. Processor 302further communicates with network interface 306, such as an Ethernet orwireless data connection, which in turn communicates with one or morenetworks 120, 122, and 254, such as the Internet or other public orprivate networks.

Processor 302 also communicates with the cloud management system 102, toexecute control logic and allow perform the management processes asdescribed above and below. Other configurations of the computing system300, associated network connections, and other hardware and softwareresources are possible.

While FIG. 3 illustrates the computing system 300 as a standalone systemincluding a combination of hardware and software, the computing system300 can include multiple systems operating in cooperation. The cloudmanagement system 102 can be implemented as a software application orprogram capable of being executed by the computing system 300, asillustrated, or other conventional computer platforms. Likewise, thecloud management system 102 can also be implemented as a software moduleor program module capable of being incorporated in other softwareapplications and programs. Further, the cloud management system 102 canalso be implemented as a software module or program module capable ofbeing incorporated in other management software applications andprograms. In any example, the cloud management system 102 can beimplemented in any type of conventional proprietary or open-sourcecomputer language. When implemented as a software application or programcode, the cloud management system 102 can be stored in a computerreadable storage medium, such as storage 310, accessible by thecomputing system 300. During execution, a copy of the cloud managementsystem 102 can be stored in the memory 304.

FIG. 4 illustrates an exemplary flow diagram of a flexible managementprocess for a cloud computing architecture including migrating virtualmachines to external clouds, according to embodiments. In 402,processing can begin. In 404, the cloud management system 102 can beconfigured to receive rules for altering a virtual machine. For example,the cloud management system 102 can be configured to receive the rulesvia the interface 204 of the cloud master 200. The rules can specifyconditions in usage of and demands on the instantiated virtual machines202, the set of resource servers 108, and/or the computing systems 110.Additionally, the rules can specify actions to take once the virtualmachines 202 and/or computing resources meet the specified conditions.The actions can include adding new virtual machines 202, removing one ormore of the virtual machines 202, migrating the virtual machines 202 toone or more of the set of resource servers 108, and/or the computingsystems 110 with additional computing resources, migrating the virtualmachines 202 to one or more of the set of resource servers 108, and/orthe computing systems 110 with less computing resources, and the like.

In 406, the cloud management system 102 can monitor for the occurrenceof the conditions specified in the rules. For example, the cloud master200, via the interface 204, can monitor for the occurrence of thecondition. The cloud master 200 can be linked to internal monitoringagents 206 and/or external monitoring agent 208. The internal monitoringagents 206 can be tools and plug-ins provided by the cloud managementsystem 102 to monitor the usage of and the demands on the set ofresource servers 108, the computing systems 110, and/or the instantiatedvirtual machines 202.

In 408, the cloud management system 102 can migrate the virtual machineto an external cloud 250 upon the occurrence of the condition asspecified in the rules. For example, the cloud management system 102 candirectly instantiate the virtual machines on the computing resources 256of the external cloud 250. For instance, the cloud management system 102can communicate with the independent cloud management system 252 and theindependent cloud management system 252 can provide the identificationand information for the computing resources 256 that will support thevirtual machines and the cloud management scheme utilized to instantiatevirtual machines. Additionally, for instance, the cloud managementsystem 102 can search the inventory 124 for the record of the availablecomputing resources 256 and the cloud management scheme. As such, thecloud management system 102 can directly instantiate the virtualmachines on the computing resources 256, as described above. Likewise,for example, the cloud management system 102 can provide the virtualmachines and/or details of the virtual machines to the independent cloudmanagement system 252, and the independent cloud management system 252can instantiate the virtual machine.

Then, in 410, the process can end, but the process can return to anypoint and repeat.

Certain embodiments may be performed as a computer application orprogram. The computer program may exist in a variety of forms bothactive and inactive. For example, the computer program can exist assoftware program(s) comprised of program instructions in source code,object code, executable code or other formats; firmware program(s); orhardware description language (HDL) files. Any of the above can beembodied on a computer readable medium, which include computer readablestorage devices and media, and signals, in compressed or uncompressedform. Exemplary computer readable storage devices and media includeconventional computer system RAM (random access memory), ROM (read-onlymemory), EPROM (erasable, programmable ROM), EEPROM (electricallyerasable, programmable ROM), and magnetic or optical disks or tapes.Exemplary computer readable signals, whether modulated using a carrieror not, are signals that a computer system hosting or running thepresent teachings can be configured to access, including signalsdownloaded through the Internet or other networks. Concrete examples ofthe foregoing include distribution of executable software program(s) ofthe computer program on a CD-ROM or via Internet download. In a sense,the Internet itself, as an abstract entity, is a computer readablemedium. The same is true of computer networks in general.

While the teachings has been described with reference to the exemplaryembodiments thereof, those skilled in the art will be able to makevarious modifications to the described embodiments without departingfrom the true spirit and scope. The terms and descriptions used hereinare set forth by way of illustration only and are not meant aslimitations. In particular, although the method has been described byexamples, the steps of the method may be performed in a different orderthan illustrated or simultaneously. Furthermore, to the extent that theterms “including”, “includes”, “having”, “has”, “with”, or variantsthereof are used in either the detailed description and the claims, suchterms are intended to be inclusive in a manner similar to the term“comprising.” As used herein, the term “one or more of” with respect toa listing of items such as, for example, A and B, means A alone, Balone, or A and B. Those skilled in the art will recognize that theseand other variations are possible within the spirit and scope as definedin the following claims and their equivalents.

1. A method comprising: instantiating, by a cloud management computingdevice that is managing a first cloud, a virtual machine on a computingdevice of the first cloud according to a request from a user; receiving,by the cloud management computing device, a rule defined by the user,wherein the rule specifies a condition that triggers a migration of thevirtual machine; receiving, from the user, a specification of amonitoring agent to monitor the virtual machine; directing themonitoring agent to monitor for the occurrence of the condition;communicating with an independent cloud management computing device thatmanages a second cloud external to the first cloud to determine:available computing resources in the second cloud; and a configurationmanagement scheme utilized by the independent cloud management computingdevice to instantiate and communicate with virtual machines on thesecond cloud; and when the monitoring agent determines that themonitored condition occurred, migrating, by the cloud managementcomputing device, the virtual machine to another computing device of thesecond cloud based on the available computing resources in the secondcloud and per the configuration management scheme utilized by theindependent cloud management computing device.
 2. The method of claim 1,wherein migrating the virtual machine to the another computing device ofthe second cloud, comprises: instantiating the virtual machine on theanother computing device of the second cloud.
 3. The method of claim 1,wherein migrating the virtual machine to the another computing device ofthe second cloud, comprises: providing the virtual machine or details ofthe virtual machine to the independent cloud management computingdevices of the second cloud; and providing a request to the independentcloud management computing device to instantiate the virtual machine onthe another computing device of the second cloud.
 4. The method of claim1, wherein the condition comprises at least one of demand on the virtualmachine, usage of the virtual machine, demand on the computing system,and usage of the computing system.
 5. The method of claim 1, whereinmonitoring for the condition comprises receiving data indicating theoccurrence of the condition.
 6. The method of claim 1, furthercomprising: generating an interface to receive the rule.
 7. The methodof claim 1, wherein the rule is formatted in a text-based language.
 8. Asystem comprising: a network interface to a set of computing devices ina first cloud and a second set of computing devices in a second cloud,wherein the first and second clouds are external to each other andmanaged by different cloud computing management devices; a memorycontaining instructions that implement a cloud management system in thecloud computing management device of the first cloud; and a processorcommunicatively connected with the network interface and the memory, theprocessor executing the instructions that implement the cloud managementsystem, which is configured to perform operations comprising:instantiating a virtual machine on the set of computing devices in thefirst cloud according to a request from a user; receiving a rule definedby the user, wherein the rule specifies a condition that triggers amigration of the virtual machine; receiving, from the user, aspecification of a monitoring agent to monitor the virtual machine;directing the monitoring agent to monitor for the occurrence of thecondition; communicating with an independent cloud management computingdevice that manages a second cloud external to the first cloud todetermine: available computing resources in the second cloud; and aconfiguration management scheme utilized by the independent cloudmanagement computing device to instantiate and communicate with virtualmachines on the second cloud; and when the monitoring agent determinesthat the monitored condition occurred, migrating the virtual machine toone of the computing devices in the second cloud based on the availablecomputing resources in the second cloud and per the configurationmanagement scheme utilized by the independent cloud management computingdevice.
 9. The system of claim 8, wherein migrating the virtual machineto the one of the computing devices in the second cloud, comprises:instantiating the virtual machine on the one of the computing devices inthe second cloud.
 10. The system of claim 8, wherein migrating thevirtual machine to the one of the computing devices in the second cloud,comprises: providing the virtual machine or details of the virtualmachine to the independent cloud computing management device of thesecond cloud; and providing a request to the independent cloud computingmanagement device of the second cloud to instantiate the virtual machineon the one of the computing devices in the second cloud.
 11. The systemof claim 8, wherein the condition comprises at least one of demand onthe virtual machine, usage of the virtual machine, demand on the set ofcomputing systems, and usage of the set of computing systems.
 12. Thesystem of claim 8, wherein monitoring for the condition comprisesreceiving data indicating the occurrence of the condition.
 13. Thesystem of claim 8, the cloud management system being further configuredto— generate an interface to receive the rule.
 14. The system of claim8, wherein the rule is formatted in a text-based language.
 15. Anon-transitory computer readable storage medium comprising instructionsfor causing a processing system to perform a method comprising:instantiating, by a cloud management computing device that is managing afirst cloud, a virtual machine on a computing device of the first cloudaccording to a request from a user; receiving, by the cloud managementcomputing device, a rule defined by the user, wherein the rule specifiesa condition that triggers a migration of the virtual machine; receiving,from the user, a specification of a monitoring agent to monitor thevirtual machine; directing the monitoring agent to monitor for theoccurrence of the condition; communicating with an independent cloudmanagement computing device that manages a second cloud external to thefirst cloud to determine: available computing resources in the secondcloud; and a configuration management scheme utilized by the independentcloud management computing device to instantiate and communicate withvirtual machines on the second cloud; and when the monitoring agentdetermines that the monitored condition occurred, migrating, by thecloud management computing device, the virtual machine to anothercomputing device in the second cloud based on the available computingresources in the second cloud and per the configuration managementscheme utilized by the independent cloud management computing device.16. The non-transitory computer readable storage medium of claim 15,wherein migrating the virtual machine to the another computing device ofthe second cloud, comprises: instantiating the virtual machine on theanother computing device of the second cloud.
 17. The non-transitorycomputer readable storage medium of claim 15, wherein migrating thevirtual machine to the another computing device of the second cloud,comprises: providing the virtual machine or details of the virtualmachine to the independent cloud management computing device of thesecond cloud; and providing a request to the independent cloudmanagement computing device to instantiate the virtual machine on theanother computing device of the second cloud.
 18. The non-transitorycomputer readable storage medium of claim 15, wherein the conditioncomprises at least one of demand on the virtual machine, usage of thevirtual machine, demand on the computing system, and usage of thecomputing system.
 19. The non-transitory computer readable storagemedium of claim 15, wherein monitoring for the condition comprisesreceiving data indicating the occurrence of the condition.
 20. Thenon-transitory computer readable storage medium of claim 15, furthercomprising: generating an interface to receive the rule.
 21. Thenon-transitory computer readable storage medium of claim 15, wherein therule is formatted in a text-based language.